Custom Feature Descriptor:

The feature descriptor that was implemented is similar to the SIFT feature descriptor. A 16x16 pixel area is first rotated to the orientation of the feature. It is then divided up into 16 smaller 4x4 pixel area around the center point. Then each pixel within the 4x4 area is sampled. The Harris value is sampled along with the direction of the larger eigenvector. The value is then binned into one of the eight bins based on its direction. The histogram bin is then reordered so that the bin with the highest value is the first feature point. The 8 bins are then pushed onto the feature data as data. The same process is then repeated for the other 15 smaller 4x4 area.

Major Design Decisions:

There exists an internal threshold included in the ratio test of 0.85 to differentiate the ratio matching from the SSD matching. Ratio matching with no thresholds would be the same as SSD matching.

The eigenvalue was used over the value obtained from using the Harris operator as it would also give the eigenvectors, allowing us to find the direction of the eigenvectors. Using eigenvalues also would make feature detection more accurate as the Harris operator is only a quick approximation of finding the minimum eigenvalue.

A threshold for feature detection of 0.05 was chosen so that there was not too many features that the program would need to search through. A rule of a local maximum must also be 0.03 above its neighboring features was also implemented to limit the features to “good” features. This however has a drawback on blurry images since the Harris values will be extremely close together. For example, bikes 4, 5, and 6 did not have any “good” features being detected.

Yosemite:

$Description: Description: Description: Z:\comp vis 6670\Features\Debug\Yosemite\plot.roc.all.png$

$Description: Z:\comp vis 6670\Features\Debug\Yosemite\plot.threshold.MOPS.png$ $Description: Z:\comp vis 6670\Features\Debug\Yosemite\plot.threshold.simple.png$ $Description: Z:\comp vis 6670\Features\Debug\Yosemite\plot.threshold.custom.png$ $Description: Description: Description: C:\Documents and Settings\acc269\Desktop\Turn in\Yosemite_harris.jpg$

Graf:

$Description: Description: Description: Z:\comp vis 6670\Features\Debug\Copy of graf\plot.roc.all.png$

$Description: Z:\comp vis 6670\Features\Debug\graf\plot.threshold.custom.png$ $Description: Z:\comp vis 6670\Features\Debug\graf\plot.threshold.MOPS.png$ $Description: Z:\comp vis 6670\Features\Debug\graf\plot.threshold.simple.png$

$Description: Description: Description: C:\Documents and Settings\acc269\Desktop\Turn in\graf_harris.jpg$

Average AUC:

Simple - Yosemite
SSD	Ratio
0.915036	0.924173

MOPS - Yosemite
SSD	Ratio
0.858670	0.901425

Custom - Yosemite
SSD	Ratio
0.740878	0.919243

Simple Bikes	SSD	Ratio
img2	0.530055	0.676195
img3	0.439059	0.570295
img4	N/A	N/A
img5	N/A	N/A
img6	N/A	N/A

MOPS Bikes	SSD	Ratio
img2	0.872762	0.859909
img3	0.919651	0.909061
img4	N/A	N/A
img5	N/A	N/A
img6	N/A	N/A

Custom Bikes	SSD	Ratio
img2	0.126733	0.760398
img3	0.154463	0.311367
img4	N/A	N/A
img5	N/A	N/A
img6	N/A	N/A

Simple Graf	SSD	Ratio
img2	0.652184	0.725156
img3	0.520969	0.684207
img4	0.505957	0.575614
img5	0.698638	0.707634
img6	0.614448	0.596050

MOPS Graf	SSD	Ratio
img2	0.851484	0.891169
img3	0.755615	0.728132
img4	0.752487	0.769758
img5	0.648936	0.704372
img6	0.515245	0.577261

Custom Graf	SSD	Ratio
img2	0.603279	0.777686
img3	0.497744	0.616061
img4	0.227873	0.462592
img5	0.134631	0.580292
img6	0.204490	0.334951

Simple Leuven	SSD	Ratio
img2	0.652184	0.528871
img3	0.189147	0.687597
img4	0.683021	0.275083
img5	1.#IND00	1.#IND00
img6	0.063396	0.938809

MOPS Leuven	SSD	Ratio
img2	0.827642	0.878467
img3	0.787131	0.835140
img4	0.868815	0.901795
img5	0.859291	0.923832
img6	0.798972	0.850638

Custom Leuven	SSD	Ratio
img2	0.445142	0.771603
img3	0.519831	0.799341
img4	0.524238	0.695739
img5	0.534493	0.744948
img6	0.421147	0.656866

Simple Wall	SSD	Ratio
img2	0.507249	0.685150
img3	0.451834	0.688991
img4	0.481637	0.697615
img5	0.426469	0.639850
img6	0.496275	0.625092

MOPS Wall	SSD	Ratio
img2	0.904111	0.893361
img3	0.887506	0.889224
img4	0.834458	0.820831
img5	0.828539	0.794064
img6	0.731961	0.713326

Custom Wall	SSD	Ratio
img2	0.550065	0.775204
img3	0.478094	0.740982
img4	0.519620	0.717042
img5	0.444460	0.646076
img6	0.606639	0.609098

Strengths and Weaknesses:

The strength of the feature detection that we implemented is that it is mostly illumination invariant. It is true that it will be weaker under dim lights as the features do not have as much contrast as when it is under a well lit condition. The feature matching is fairly rotationally invariant because a Gaussian window is used in determining if a pixel is a feature or not and most descriptors are rotate the sampling area based on the orientation of the feature. However, this also needs to make sure the feature detection detects the orientation accurately as most feature descriptor will be very dependent on the orientation of the feature.

The feature detection and matching is good when there are many areas of the images that have high contrast features. Images with low contrasts such as the blurred bike images will have local maximas below the threshold and thus will often be discarded as a feature. This results in less point for feature matching and will thus have higher chances of failed matches. However, a very low maxima will introduce too many points, making feature matching an extremely slow process.

The feature detection and matching that we implemented is also weak in detecting repeating features that closely resembles each other, especially if the matched images are warped by viewing it from different perspectives. As is seen in Walls.